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Abstract 

Over  the  period  of  this  contract,  William  Gropp  has  investigated  several  aspects  of  adaptive 
methods  for  partial  differential  equations  on  vector  and  parallel  computers.  The  work  has  included 
the  porting  of  several  adaptive  codes  to  run  in  parallel  on  several  parallel  machines  and  the  com¬ 
parison  of  various  domain  decomposition  algorithms  for  elliptic  partial  differential  equations.  A 
new  technique  for  mapping  mesh  points  to  processors  in  a  static  way  has  been  developed;  this 
takes  advantage  of  the  structure  of  the  family  of  solutions  without  singling  out  any  one  solution. 
Experience  with  the  parallel  codes  has  shown  that  good  efficiencies  can  be  reached  without  a  dis¬ 
proportionate  amount  of  work;  however,  data  communication  times  are  important  and  can  reduce 
the  benefit  of  using  parallel  computers.  Early  results  with  domain  decomposition  methods  indicate 
that  they  do  work  well  on  parallel  computers,  but  they  don’t  have  an  overwhelming  advantage  over 
more  conventional  approaches. 


1.  Research  Objectives 

The  purpose  of  this  work  is  to  study  the  application  of  a  particular  adaptive  method,  Local 
Uniform  Mesh  Refinement  (LUMR)  to  vector  and  parallel  computers.  The  specific  areas  to  be 
studied  were 

•  Solution  of  elliptic  equations. 

•  Where  and  how  to  refine,  preserving  the  effectiveness  of  vector  and  parallel  computers. 

•  Programming  and  data  structures. 

Some  of  the  work  is  also  applicable  to  other  adaptive  methods;  it  is  hoped  that  these  results  may 
point  out  how  other  adaptive  techniques  may  be  used  on  vector  and  parallel  computers. 

2.  Progress 

2.1.  LUMR  on  parallel  processors 

A  version  of  the  LUMR  code  which  handles  the  data  structure  manipulations  was  implemented 
and  tested  on  the  Intel  iPSC-d7  hypercube  which  was  acquired  by  the  Computer  Science  Depart¬ 
ment  at  Yale.  These  experiments  showed  that,  just  as  in  vector  computing,  there  is  a  minimum 
size  or  granularity  below  which  various  overheads  become  dominant.  The  results  were  presented  in 
a  technical  report,  to  appear  in  a  SIAM  proceedings. 

An  analysis  of  the  time  complexity  of  running  a  2-d  adaptive  mesh  refinement  code  on  a 
loosely-coupled  parallel  computer  was  developed.  The  model  of  a  parallel  computer  used  included 
data  communication  delays;  these  terms  proved  crucial  in  predicting  the  observed  performance. 
This  analysis  allowed  suggested  specific  implementations  of  certain  operations  in  the  code.  To  test 
these  results,  a  2-d  version  of  a  Local  Uniform  Mesh  Refinement  code  was  developed  and  ported 
to  a  ring  of  Apollo  Workstations.  The  results  of  these  tests  validated  the  theory,  and  suggest  that 
loosely-coupled  parallel  processors  will  work  even  for  adaptive  techniques  if  they  are  designed  with 
enough  communications  bandwidth.  Further,  it  was  relatively  easy  to  move  a  properly  designed 
serial  code  to  a  parallel  processor.  This  suggests  that,  once  the  newness  of  parallel  computing 
fades,  programming  a  parallel  computer  may  not  be  much  harder  than  a  single  computer  as  long 
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as  some  care  is  taken  in  the  initial  design.  William  Gropp’s  results  give  a  way  to  determine  what 
is  enough  for  these  applications.  These  results  were  presented  in  a  technical  report,  and  submitted 
for  publication  to  Computers  and  Mathematics. 

Another  result  of  these  studies  was  an  appreciation  that  a  memory  access  model  serves  as  a 
good  way  to  view  both  message  passing  and  shared  memory  parallel  computers.  This  is  similar  to 
the  way  in  which  algorithms  for  vector  computers  have  been  discovered  by  optimizing  the  memory 
access  patterns  in  existing  techniques.  This  was  discussed  by  William  Gropp  at  a  workshop,  and 
will  appear  in  the  proceedings  of  that  meeting. 

2.2.  Domnin  Decomposition 

<  >nci  dillicult  part  of  solving  partial  diHdrenlin!  equations  in  parallel  is  solving  the  linear  systems 
that  arise  from  implicit  discretizations.  With  Dr.  David  Keyes,  William  Gropp  lias  been  evaluating 
most  of  the  published  methods  of  Domain  Decomposition,  examining  their  relative  performance 
and  their  suitability  for  real  (rather  than  theoretical)  parallel  computers.  The  early  results  of  this 
work  show  that  these  techniques  are  indeed  applicable  to  the  problem  of  solving  linear  systems  on 
parallel  computers,  but  do  not  offer  a  major  advance,  particularly  on  general  problems. 

2.3.  Mesh  distribution 

With  Dr.  Ilse  Ipsen,  William  Gropp  has  been  investigating  a  new  method  for  distributing 
work  among  parallel  processors.  This  work  is  based  on  viewing  an  adaptive  method  for  partial 
differential  equations  as  a  graph  problem,  where  the  graphs  are  not  arbitrary,  but  rather  have 
a  certain  locality  based  on  the  continuity  of  the  solution.  The  method  is  a  new  form  of  Gray- 
code,  called  an  interleaved  Gray-code,  which  allows  easy  labeling  of  graph  nodes  even  when  the 
maximal  level  of  refinement  is  unknown,  allows  easy  determination  of  nearby  nodes  in  the  graph,  is 
completely  deterministic,  and  often  (in  a  well-defined  sense)  distributes  the  graph  efficiently  across 
a  hypercube.  This  work  has  been  presented  at  an  international  meeting  on  parallel  computing,  and 
will  appear  as  a  technical  report. 
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3.  Publications  and  reports 
Publications 

D.  E.  Keyes  and  W,  D.  Gropp,  A  Comparison  of  Domain  Decomposition  Techniques  for  Elliptic 
Partial  Differential  Equations  and  their  Parallel  Implementation,  SIAM  Journal  on 
Scientific  and  Statistical  Computing,  8/2  March  (1987),  pp.  sl66-s202. 

W.  D.  Gropp  and  D.  E.  Keyes,  Complexity  of  Parallel  Implementation  of  Domain  Decomposi¬ 
tion  Techniques  for  Elliptic  Partial  Differential  Equations,  1986.  To  appear  in  SIAM 
Journal  on  Scientific  and  Statistical  Computing. 

W.  D.  Gropp,  Local  Uniform  Mesh  Refinement  on  Loosely-Coupled  Parallel  Processors,  1987. 
Submitted  to  Computers  and  Mathematics. 

Proceedings 

W.  D.  Gropp,  Dynamic  Grid  Manipulation  for  PDEs  on  Hypercube  Parallel  Processors,  Pre¬ 
sented  at  the  ARO  workshop  on  Parallel  Processing  and  Medium  Scale  Multiprocess¬ 
ing,  Jan.  1986,  to  appear  as  a  conference  proceedings. 

W.  D.  Gropp,  Adaptive  methods  for  hyperbolic  problems  on  local  memory  parallel  processors, 
Paper  presented  at  November  3-7,  1986  Minnesota  Workshop  on  Numerical  Algo¬ 
rithms  for  Modern  Parallel  Computer  Architectures,  to  appear  in  proceedings  of  meet¬ 
ing. 

Technical  reports 

W.  D.  Gropp,  Dynamic  Grid  Manipulation  for  PDEs  on  Hypercube  Parallel  Processors,  Tech¬ 
nical  Report  YALE/DCS/RR-458,  Yale  University,  Department  of  Computer  Science, 
March  1986. 

W.  D.  Gropp  and  I.  C.  F.  Ipsen,  Performance  Comparison  of  Adaptive  PDE  algorithms  on 
Various  Hypercube  Architectures,  1986.  Talk  given  at  International  Conference  on 
Vector  and  Parallel  Computing  at  Loeri,  Norway  in  preparation  as  a  Harwell  report. 

Presentations 

W.  D.  Gropp,  Local  Uniform  Mesh  Refinement  on  Vector  and  Parallel  Processors,  Minisym¬ 
posium  presentation  at  SEG-SIAM-SPE  1985  Meeting  on  Seismic  Exploration  and 
Reservoir  Modeling. 

W.  D.  Gropp,  Adaptive  Mesh  Refinement  for  PDEs  in  S-D  on  Parallel  Processors,  Presented 
at  the  Second  SIAM  conference  on  Parallel  Processing  for  Scientific  Computing,  1985, 
Norfolk,  VA. 

W.  D.  Gropp  and  I.  C.  F.  Ipsen,  Embedding  Binary  Trees  into  a  hypercube,  Presented  at  the  Sec¬ 
ond  SIAM  conference  on  Parallel  Processing  for  Scientific  Computing,  1985,  Norfolk, 
VA. 

W.  D.  Gropp,  Hypercube  Research  at  Yale,  Invited  presentation  at  Second  Conference  on  Hy¬ 
percube  Multiprocessors,  Knoxville,  TN,  Sept.  29-Oct.  1,  1986. 
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